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Highlights  of  GAO-06-368,  a  report  to 
congressional  committees 


DEFENSE  ACQUISITIONS 

Major  Weapon  Systems  Continue  to 
Experience  Cost  and  Schedule  Problems 
under  DOD's  Revised  Policy 


Why  GAO  Did  This  Study 

The  Department  of  Defense  (DOD) 
is  planning  to  invest  $1.3  trillion 
between  2005  and  2009  in 
researching,  developing,  and 
procuring  major  weapon  systems. 
How  DOD  manages  this  investment 
has  been  a  matter  of  congressional 
concern  for  years.  Numerous 
programs  have  been  marked  by 
cost  overruns,  schedule  delays,  and 
reduced  performance.  Over  the 
past  3  decades,  DOD’s  acquisition 
environment  has  undergone  many 
changes  aimed  at  curbing  cost, 
schedule,  and  other  problems.  In 
order  to  determine  if  the  policy 
DOD  put  in  place  is  achieving  its 
intended  goals,  we  assessed  the 
outcomes  of  major  weapons 
development  programs  initiated 
under  the  revised  policy. 
Additionally,  we  assessed  whether 
the  policy’s  knowledge-based, 
evolutionary  principles  are  being 
effectively  implemented,  and 
whether  effective  controls  and 
specific  criteria  are  in  place  and 
being  used  to  make  sound 
investment  decisions. 


What  GAO  Recommends 


GAO  recommends  that  DOD  insert 
specific  criteria  into  the  policy  at 
key  investment  points  and  require 
programs  satisfy  those  criteria 
before  allowing  them  to  move 
forward.  In  order  to  insure 
transparency  and  accountability, 
GAO  also  recommends  that  DOD 
require  decision  makers  to  include 
the  rationale  for  their  decisions  in 
decision  documentation.  DOD 
partially  concurred  with  our 
recommendations. 

www.gao.gov/cgi-bin/getrpt7GAO-06-368. 

To  view  the  full  product,  including  the  scope 
and  methodology,  click  on  the  link  above. 
For  more  information,  contact  Michael  J. 
Sullivan  at  (202)  512-4841  or 
sullivanm@gao.gov. 


What  GAO  Found 

Changes  made  in  DOD’s  acquisition  policy  over  the  past  5  years  have  not 
eliminated  cost  and  schedule  problems  for  major  weapons  development 
programs.  Of  the  23  major  programs  we  assessed,  10  are  already  expecting 
development  cost  overruns  greater  than  30  percent  or  have  delayed  the 
delivery  of  initial  operational  capability  to  the  warfighter  by  at  least  1  year. 
The  overall  impact  of  these  costly  conditions  is  a  reduction  in  the  value  of 
DOD’s  defense  dollars  and  a  lower  return  on  investment.  The  following  table 
illustrates  the  problem. 


Cost  and  Schedule  Outcomes  Sorted  by  Percent  of  Product  Development  Remaining 

Programs 

Percent  cost 
growth" 

Schedule  growth, 
in  months 

Percent  of 

development 

remaining 

Aerial  Common  Sensor 

45% 

24 

85% 

Future  Combat  System 

48% 

48 

78% 

Joint  Strike  Fighter 

30% 

23 

60% 

Expeditionary  Fighting  Vehicle 

61% 

48 

49% 

C-130  Avionics  Modernization 

Program 

122% 

Delays  anticipated 

Undetermined 

Global  Hawk  (RQ-4B) 

166% 

Delays  anticipated 

Undetermined 

Sources:  DOD  (data);  GAO  (analysis  and  presentation). 


“Cost  growth  is  expressed  as  the  percent  change  in  program  development  cost  estimates  in 
2005  base  year  dollars. 

Poor  execution  of  the  revised  acquisition  policy  is  a  major  cause  of  DOD’s 
continued  problems.  DOD  frequently  bypasses  key  steps  of  the  knowledge- 
based  process  outlined  in  the  policy,  falls  short  of  attaining  key  knowledge, 
and  continues  to  pursue  revolutionary — rather  than  evolutionary  or 
incremental — advances  in  capability.  Nearly  80  percent  of  the  programs 
GAO  reviewed  did  not  fully  follow  the  knowledge-based  process  to  develop  a 
sound  business  case  before  committing  to  system  development.  Most  of  the 
programs  we  reviewed  started  system  development  with  immature 
technologies,  and  half  of  the  programs  that  have  held  design  reviews  did  so 
before  achieving  a  high  level  of  design  maturity.  These  practices  increase  the 
likelihood  that  problems  will  be  discovered  late  in  development  when  they 
are  more  costly  to  address.  Furthermore,  DOD’s  continued  pursuit  of 
revolutionary  leaps  in  capability  also  runs  counter  to  the  policy’s  guidance. 

DOD  has  not  closed  all  of  the  gaps  in  the  policy  that  GAO  identified  nearly 
3  years  ago,  particularly  with  regard  to  adding  controls  and  criteria.  Effective 
controls  require  decision  makers  to  measure  progress  against  specific 
criteria  and  ensure  that  managers  capture  key  knowledge  before  moving  to 
the  next  acquisition  phase.  However,  DOD’s  policy  continues  to  allow 
managers  to  approach  major  investment  decisions  with  many  unknowns. 
Without  effective  controls  that  require  program  officials  to  satisfy  specific 
criteria,  it  is  difficult  to  hold  decision  makers  or  program  managers 
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United  States  Government  Accountability  Office 
Washington,  DC  20548 


April  13,  2006 

The  Honorable  John  Ensign 
Chairman 

The  Honorable  Daniel  K.  Akaka 
Ranking  Minority  Member 

Subcommittee  on  Readiness  and  Management  Support 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Duncan  L.  Hunter 
Chairman 

Committee  on  Armed  Services 
House  of  Representatives 

DOD’s  planned  investment  in  research,  development,  and  procurement  of 
major  weapon  systems  will  total  approximately  $1.3  trillion  between  2005 
and  2009,  with  over  $800  billion  of  that  investment  yet  to  be  made.  DOD  is 
facing  a  significant  number  of  problems  in  managing  its  acquisitions. 
Military  operations  in  Afghanistan  and  Iraq  are  consuming  a  large  share  of 
DOD  resources  and  causing  the  department  to  invest  more  money  sooner 
than  expected  to  replace  or  fix  existing  weapons.  Meanwhile,  DOD  is 
intent  on  transforming  military  operations  while  pursuing  multiple 
megasystems  that  are  expected  to  be  the  most  expensive  and  complex 
ever.  These  costly  conditions  coupled  with  increases  in  spending  for  other 
national  priorities,  such  as  health  care  and  social  security,  make  it 
essential  that  DOD  effectively  leverage  its  investments,  particularly  in 
weapon  system  acquisitions.  If  DOD  manages  its  current  portfolio  of 
weapons  within  traditional  margins  of  error,  the  financial  consequences 
could  be  dire. 

DOD’s  strategy  for  acquiring  major  weapon  systems  has  traditionally  been 
to  plan  programs  that  would  achieve  a  big  leap  forward  in  capability 
within  a  single  development  program,  a  strategy  that  often  results  in  major 
cost  and  schedule  problems.  We  have  assessed  weapon  acquisitions  as  a 
high-risk  area  for  15  years,  and  although  U.S.  weapons  are  among  the  best 
in  the  world,  the  programs  to  acquire  them  have  continued  to  produce 
poor  cost  and  schedule  outcomes.  However,  the  current  defense 
acquisition  environment  continues  to  be  characterized  by  cost  and 
schedule  growth,  a  lack  of  confidence  by  congressional  and  DOD  leaders, 
and  no  appreciable  improvement  in  the  defense  acquisition  system.  DOD 
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knows  what  to  do  to  achieve  better  outcomes.  It  has  written  into  policy  an 
approach  that  advocates  that  adequate  knowledge  be  attained  at  critical 
junctures  before  DOD  managers  agree  to  invest  more  money  in  the  next 
phase  of  weapon  system  development.  The  policy  also  emphasizes 
evolutionary  principles  for  acquiring  weapons  rather  than  trying  to  achieve 
a  big  leap  forward  in  capability  within  a  single  development  program.  We 
have  reported  in  the  past  that  DOD’s  revised  policy  does  not  incorporate 
adequate  controls  to  ensure  the  effective  implementation  of  a  knowledge- 
based,  evolutionary  acquisition  process.  However,  DOD  believes  that  the 
policy  includes  the  necessary  controls  to  achieve  effective  outcomes. 

You  requested  that  we  evaluate  DOD’s  compliance  with  and 
implementation  of  its  revised  acquisition  policy  intended  to  produce  better 
cost,  schedule,  and  performance  outcomes  for  major  acquisition 
programs.  In  order  to  obtain  an  early  assessment  of  the  cost  and  schedule 
impact  of  the  revised  policy,  and  to  assess  DOD’s  effectiveness  in 
implementing  a  knowledge-based,  evolutionary  acquisition  approach  we 
assessed  (1)  the  cost  and  schedule  status  of  major  weapons  development 
programs  initiated  under  the  revised  policy,  (2)  whether  the  policy’s 
knowledge-based,  evolutionary  acquisition  principles  are  being  effectively 
implemented,  and  (3)  whether  effective  controls  and  specific  criteria  are 
in  place  and  being  used  to  make  sound  investment  decisions. 

In  conducting  our  evaluation,  we  reviewed  pertinent  acquisition  statutes, 
policies,  and  guidance;  analyzed  development  cost  and  schedule  data  for 
23  major  acquisition  programs  approved  to  start  system  development 
under  DOD’s  revised  acquisition  policy  between  October  2000  and 
December  2004;  conducted  case  study  reviews  of  nine  of  those 
23  programs;  and  interviewed  officials  from  the  Office  of  Secretary  of 
Defense  and  each  of  the  military  services.  We  conducted  our  review  from 
May  2005  to  February  2006  in  accordance  with  generally  accepted 
government  auditing  standards.  Additional  information  about  our 
methodology  is  contained  in  appendix  I. 


Results  in  Brief 


DOD’s  revised  acquisition  policy  has  not  led  to  improved  acquisition 
program  outcomes.  Programs  initiated  under  the  revised  policy  are 
already  beginning  to  experience  cost  and  schedule  problems  similar  to 
programs  managed  under  prior  versions  of  the  policy.  Although  the 
programs  we  reviewed  have  been  in  development  for  only  a  short  period 
of  time,  nearly  half  are  already  estimating  development  cost  growth 
greater  than  30  percent  or  are  expecting  to  delay  initial  delivery  to  the 
warfighter  by  at  least  1  year.  Program  officials  are  facing  the  familiar 
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predicament  of  having  to  add  unplanned  money  or  time  or  to  reduce 
system  capabilities  and  quantities  after  initial  business  cases  have  been 
approved  and  system  development  has  begun.  As  a  result,  DOD  is  reducing 
its  buying  power  and  is  not  achieving  the  return  on  investment  it  expected 
when  the  programs  began. 

Acquisition  officials  are  not  effectively  implementing  the  revised 
acquisition  policy’s  knowledge-based  process.  They  regularly  bypass  key 
phases  of  the  early  acquisition  process,  approach  key  decision  points  with 
limited  knowledge  about  critical  technologies  and  system  design,  and  do 
not  employ  evolutionary  acquisition  principles.  Nearly  80  percent  of  the 
programs  we  assessed  were  permitted  to  bypass  the  policy’s  initial  major 
decision  review  and  the  associated  systems-engineering  process  that  are 
intended  to  ensure  that  a  system’s  requirements  match  available  resources 
and  that  a  sound  business  case  is  developed  prior  to  starting  system 
development.  By  not  consistently  following  key  processes  and  strategies, 
acquisition  officials  are  not  ensuring  that  a  solid  foundation  of  knowledge 
about  cost,  schedule,  and  performance  is  established  before  allowing 
programs  to  start  system  development,  thus  resulting  in  unexecutable 
business  cases.  Although  the  policy  explicitly  states  that  programs  shall 
increase  program  knowledge  by  maturing  technologies  before  beginning 
system  development,  we  found  that  almost  three-fourths  of  the  programs 
started  since  the  policy  was  revised  began  development  with  immature 
critical  technologies.  Our  analysis  also  indicates  that  decision  makers  are 
continuing  to  commit  programs  to  system  demonstration  and  initial 
manufacturing  before  officials  have  demonstrated  high  levels  of  design 
knowledge,  as  emphasized  in  the  policy.  In  addition,  programs  like  the 
Joint  Strike  Fighter  and  Future  Combat  System  are  still  structured  to 
achieve  major  leaps  in  capability  within  a  single  development  program,  a 
strategy  that  has  historically  proven  to  be  problematic  in  terms  of  cost  and 
schedule  outcomes. 

Effective  implementation  of  the  revised  policy  is  limited  by  the  absence  of 
effective  controls  that  require  compliance  and  specific  criteria  for  clearly 
demonstrating  that  acceptable  levels  of  knowledge  about  technology, 
design,  and  manufacturing  have  been  attained  at  critical  junctures  during 
system  development  before  making  further  investments  in  a  program. 
Without  effective  controls,  the  policy  cannot  prevent  DOD  decision 
makers  from  starting  system  development  even  when  they  face  significant 
unknowns  about  technology,  design,  and  production.  Without  specific 
criteria — or  standards  against  which  a  judgment  or  decision  is  quantifiably 
based — decision  makers  are  permitted  to  make  decisions  on  the  basis  of 
subjective  judgment.  We  reported  this  condition  in  2003,  yet  DOD  has  not 
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closed  gaps  in  the  policy.  In  the  absence  of  such  controls  and  criteria, 

DOD  faces  the  added  problems  of  transparency  and  accountability 
because  it  often  does  not  sufficiently  document  the  rationale  for  its 
decisions  to  allow  acquisition  programs  to  advance  with  low  levels  of 
technology,  design,  and  manufacturing  knowledge. 

This  report  contains  recommendations  that  DOD  require  programs  to  meet 
specific  knowledge-based  criteria  at  each  key  decision  points  in  the 
acquisition  process  and  require  decision  makers  to  provide  clear  and 
specific  rationale  for  their  decisions.  In  addition,  we  recommend  that 
before  programs  enter  system  development  they  should  be  required  to 
complete  disciplined  concept  and  technology  development  phases  that 
include  specific  activities  dedicated  to  capturing  knowledge  critical  to 
developing  an  executable  business.  DOD  partially  concurred  with  our 
recommendations.  DOD  agrees  that  knowledge-based  decision  making  is 
consistent  with  sound  business  practice  and  stated  that  it  would  continue 
to  develop  policy  that  reflects  a  knowledge-based  approach  and  improves 
acquisition  outcomes.  DOD  also  agrees  that  acquisition  decisions  should 
be  documented,  decision  makers  should  be  held  accountable,  and  that 
they  should  provide  the  rationale  for  their  decisions. 


Background 


Historically,  DOD’s  programs  for  acquiring  major  weapon  systems  have 
taken  longer,  cost  more,  and  often  delivered  fewer  quantities  and  other 
capabilities  than  planned.  GAO  has  documented  these  problems  for 
decades.  In  1970,  GAO  reported  that  considerable  cost  growth  had  been 
and  was  continuing  to  occur  on  many  current  development  programs. 

Since  that  report  was  issued,  numerous  changes  have  been  made  to  DOD’s 
acquisition  process  and  environment  to  try  to  improve  acquisition 
outcomes.  Those  changes  include  numerous  executive  branch  initiatives 
and  legislative  actions  as  well  as  roughly  11  revisions  to  DOD’s  acquisition 
policy  between  1971  and  2005.  Despite  these  efforts,  defense  acquisition 
programs  in  the  past  3  decades  continued  to  routinely  experience  cost 
overruns,  schedule  slips,  and  performance  shortfalls. 

Figure  1  illustrates  the  continued  problem  of  development  cost  overruns. 
The  figure  depicts  the  combined  cost  overruns  for  large  development 
programs  (programs  totaling  more  than  $1  billion  for  research, 
development,  testing  and  evaluation  in  fiscal  year  2005  dollars)  in  each  of 
the  past  3  decades.  The  figure  also  identifies  some  of  the  major  studies  and 
improvement  efforts  initiated  during  this  time  frame.  As  the  figure 
illustrates,  efforts  to  improve  acquisition  outcomes  have  not  been 
successful  in  curbing  acquisition  cost  problems.  Programs  initiated  in  the 
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1970s  exceeded  DOD’s  initial  investment  estimate  by  30  percent,  or 
$13  billion  (in  fiscal  year  2005  dollars),  and  similar  outcomes  continued 
during  the  subsequent  decades  despite  numerous  reform  efforts  and  policy 
revisions. 


Figure  1 :  Development  Cost  Overruns  by  Decade  (in  Fiscal  Year  2005  Dollars)  and  Key  Reform  Efforts 


1970-1979 

1980  -1989 

1990-1999 

Development 

Development 

Development 

cost  overrun: 

cost  overrun: 

cost  overrun: 

$13  billion 

$12  billion 

$15  billion 

(30%) 

(39%) 

(40%) 

Key  Studies  and  Initiatives  Impacting  the  Defense  Acquisition  Process 

•  1970  Fitzhugh  Commission 

•  1981  Carlucci  Initiatives 

•  1994  Federal  Acquisition  Streamlining  Act 

•  1972  Commission  on  Government 

•  1982  Grace  Commission 

•  1996  Clinger-Cohen  Act 

Procurement 

1986  Packard  Commission 

DOD  Acquisition  Policy  Changes 

•  1971  DOD  5000  policy  established 

•  1980  Policy  revised 

•  1991  Policy  revised 

•  1975  Policy  revised 

•  1982  Policy  revised 

•  1996  Policy  revised 

•  1 977  Policy  revised 

•  1985  Policy  revised 

•  1986  Policy  revised 

•  1987  Policy  revised 

Source:  DOD  (data);  GAO  (analysis  and  presentation). 


Since  the  mid-1990s,  we  have  studied  the  best  practices  of  leading 
commercial  companies.  Taking  into  account  the  differences  between 
commercial  product  development  and  weapons  acquisitions,  we 
articulated  a  best  practices  product  development  model  that  relies  on 
increasing  knowledge  when  developing  new  products,  separating 
technology  development  from  product  development,  and  following  an 
evolutionary  or  incremental  product  development  approach.  This 
knowledge-based  approach  requires  developers  to  make  investment 
decisions  on  the  basis  of  specific,  measurable  levels  of  knowledge  at 
critical  junctures  before  investing  more  money  and  before  advancing  to 
the  next  phase  of  acquisition.  An  evolutionary  product  development 
process  defines  the  individual  increments  on  the  basis  of  mature 
technologies  and  a  feasible  design  that  are  matched  with  firm 
requirements.  Each  increment  should  be  managed  as  a  separate  and 
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distinct  acquisition  effort  with  its  own  cost,  schedule  and  performance 
baseline.  An  increment  that  excludes  one  of  these  key  elements  puts  an 
extra  burden  on  decision  makers  and  provides  a  weak  foundation  for 
making  development  cost  and  schedule  estimates.  The  knowledge-based, 
evolutionary  approach  in  our  model  is  intended  to  help  reduce 
development  risks  and  to  achieve  better  program  outcomes  on  a  more 
consistent  basis. 

Hoping  to  improve  acquisition  outcomes,  DOD  leaders  initiated  significant 
revisions  to  the  department’s  acquisition  policy  again  in  October  2000,  by 
adopting  the  knowledge-based,  evolutionary  system  development 
approach.1  We  reported  in  November  2003,  that  much  of  the  revised  policy 
agrees  with  GAO’s  extensive  body  of  work  and  that  of  successful 
commercial  firms.  DOD’s  revised  policy  emphasizes  the  importance  of  and 
provides  a  good  framework  for  capturing  knowledge  about  critical 
technologies,  product  design,  and  manufacturing  processes.  If  properly 
implemented  and  enforced  this  approach  could  help  DOD’s  decision 
makers  gain  the  confidence  they  need  to  make  significant  and  sound 
investment  decisions  for  major  weapon  systems.  Furthermore,  the  policy’s 
emphasis  on  evolutionary  system  development  sets  up  a  more  manageable 
environment  for  achieving  knowledge.  We  also  noted  that  DOD’s  policy 
strongly  suggests  the  separation  of  technology  development  from  system 
development,  a  best  practice  that  helps  reduce  technological  risk  at  the 
start  of  a  program  and  makes  cost  and  delivery  estimates  much  more 
predictable.2  Figure  2  depicts  in  general  how  DOD’s  revised  policy  adopts 
key  aspects  of  the  best  practices  model. 


xIn  addition  to  the  acquisition  policy,  the  process  used  by  DOD  to  establish  program 
funding,  known  as  the  Planning,  Programming,  Budgeting  and  Execution  System  (PPBES), 
and  the  process  used  to  determine  system  requirements,  now  called  the  Joint  Capabilities 
Integration  and  Development  System  (JCIDS),  also  impact  program  outcomes.  Both 
processes  are  currently  being  studied  to  determine  if  any  changes  could  be  made  to 
improve  program  performance  as  it  relates  to  funding  and  requirements. 

2GAO.  Defense  Acquisitions:  DOD’s  Revised  Policy  Emphasises  Best  Practices,  but  More 
Controls  Are  Needed,  GAO-04-53  (Washington,  D.C.:  Nov.  10,  2003). 
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Figure  2:  Comparison  of  DOD’s  Revised  Policy  and  Commercial  Best  Practices  Model 
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Source:  DOD  (data);  GAO  (analysis  and  presentation). 


Although  DOD  took  significant  steps  in  the  right  direction,  its  policy  does 
not  include  controls  that  require  program  officials  to  meet  the  key  criteria 
that  we  believe  are  necessary  for  ensuring  that  acceptable  levels  of 
knowledge  are  actually  captured  before  making  additional  significant 
investments.  We  previously  recommended  that  DOD  design  and 
implement  necessary  controls  to  ensure  that  appropriate  knowledge  is 
captured  and  used  to  make  decisions  about  moving  a  program  forward 
and  investing  more  money  at  critical  junctures.  DOD  officials 
acknowledged  the  advantages  of  using  knowledge-based  controls,  but 
stated  that  they  believed  the  policy  already  included  enough  controls  to 
achieve  effective  program  results.  The  officials  agreed  to  monitor  the 
acquisition  process  to  assess  the  effectiveness  of  those  controls  and  to 
determine  whether  additional  ones  are  necessary. 
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DOD’s  Revised  Policy 
Has  Not  Improved 
Development 
Program  Outcomes 


The  cost  and  schedule  outcomes  being  achieved  by  development  programs 
initiated  since  DOD  first  issued  its  revised  policy  have  not  improved  over 
those  achieved  by  programs  managed  under  prior  versions  of  the  policy. 

Of  the  23  major  programs  we  assessed,  10  have  already  reported  estimated 
development  cost  growth  greater  than  30  percent  or  expected  delays  of  at 
least  1  year  in  delivery  of  an  initial  operational  capability  to  the  warfighter. 
These  programs  combined  represent  a  cost  increase  of  $23  billion  (in 
fiscal  year  2005  dollars)  and  an  average  delay  in  delivery  of  initial 
capability  of  around  2  years.  Most  of  the  other  programs  were  still  in  the 
early  stages  as  of  December  2005  with  over  half  of  system  development 
remaining  and  had  not  yet  reported  an  adequate  amount  of  cost  or 
schedule  data  to  effectively  analyze  their  progress.  Table  1  contains  the 
cost  and  schedule  increases  for  the  23  programs  we  assessed,  expressed 
as  a  percentage  of  each  program’s  development  estimate. 
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Table  1:  Cost  and  Schedule  Outcomes  for  23  Programs  Initiated  under  the  Revised  Policy  (as  of  December  2005) 


Program 

Percent  growth  in  estimated 
development  cost8 

Percent  growth  in  estimated 
development  schedule 

Expeditionary  Fighting  Vehicle 

61% 

70% 

Active  Electronically  Scanned  Array  radar  (upgrade 
for  F/A-18  E/F  fighter/attack  aircraft) 

14% 

1% 

Global  Flawk  unmanned  aerial  vehicle 

166% 

Undetermined 

Joint  Strike  Fighter 

30% 

23% 

UFI-60M  helicopter  upgrade 

151% 

25% 

C-130  Avionics  Modernization  Program 

122% 

Undetermined 

C-5  Reliability  Enhancement  and  Re-engining 

Program 

0% 

25% 

Joint  Tactical  Radio  System  Cluster  1 

31% 

44% 

Joint  Tactical  Radio  System  Waveform 

44% 

Undetermined 

Advanced  Anti-radiation  Guided  Missile 

7% 

0% 

Multi-Platform  Radar  Technology  Insertion  Program 

0% 

Undetermined 

Future  Combat  System 

48% 

53% 

E-2  Advanced  Hawkeye 

5% 

0% 

Warfighter  Information  Network-Tactical 

0% 

0% 

Small  Diameter  Bomb 

0% 

0% 

EA-18G 

7% 

0% 

Joint  Tactical  Radio  System  Cluster  5 

0% 

2% 

Multi-Mission  Maritime  Aircraft 

0% 

0% 

Standard  Missile-6  Extended  Range  Active  Missile 

Block  1 

0% 

0% 

Aerial  Common  Sensor 

45% 

36% 

B-2  Radar  Modernization  Program 

0% 

0% 

Patriot/Medium  Extended  Air  Defense  System 

Combined  Aggregate  Program  (fire  unit) 

0% 

0% 

Mission  Planning  System 

0% 

0% 

Sources:  DOD  (data);  GAO  (analysis  and  presentation). 

“Cost  qrowth  is  expressed  as  the  percent  chanqe  in  proqram  development  cost  estimates  in  fiscal 
year  2005  dollars. 

The  Army’s  Future  Combat  System  is  a  case  in  point.  Less  than  3  years 
after  program  initiation  and  with  $4.6  billion  invested,  the  Army  has 
already  increased  its  development  cost  estimate  $8.9  billion  or  48  percent 
and  delayed  delivery  of  initial  capability  by  4  years  over  the  original 
business  case.  Similarly,  just  over  1  year  after  initiating  development  of  the 
Aerial  Common  Sensor  aircraft,  the  Army  has  reported  that  severe  weight 
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and  design  problems  discovered  during  development  have  stopped  work 
on  the  program.  As  a  result,  program  officials  are  anticipating  at  least  a 
45  percent  cost  increase  and  a  delay  of  2  years  in  delivering  an  initial 
capability  to  the  warfighter.  These  two  Army  programs  are  not  the  only 
ones  experiencing  problems.  Table  2  contains  cost  and  schedule  data  for 
6  of  the  10  largest  development  programs  initiated  under  the  revised 
policy,  including  the  Future  Combat  System  and  Aerial  Common  Sensor. 
As  the  table  illustrates  there  are  several  programs  experiencing  large  cost 
increases  and  schedule  delays. 


Table  2:  Cost  and  Schedule  Outcomes  for  6  of  the  10  Largest  Development  Programs  Sorted  by  Percent  of  System 
Development  Remaining 

Programs 

Percent  development  cost 
growth 

Delay  in  delivery  of  initial 
capability  in  months 

Percent  of  development 
remaining 

Aerial  Common  Sensor 

45% 

24 

85% 

Future  Combat  System 

48% 

48 

78% 

Joint  Strike  Fighter 

30% 

23 

60% 

Expeditionary  Fighting  Vehicle 

61% 

48 

49% 

C-130  Avionics  Modernization 
Program 

122% 

Delays  anticipated  due  to 
program  restructure 

Undetermined  due  to  program 
restructure 

Global  Hawk  (RQ-4B) 

166% 

Delays  anticipated  due  to 
program  restructure 

Undetermined  due  to  program 
restructure 

Sources:  DOD  (data);  GAO  (analysis  and  presentation). 


A  good  measure  of  acquisition  performance  is  return  on  investment  as 
expressed  in  acquisition  program  unit  cost  because  unit  cost  represents 
the  value  DOD  is  getting  for  its  acquisition  dollars  invested  in  a  certain 
program.  The  programs  listed  in  table  2  will  not  achieve  the  return  on 
investment  that  DOD  anticipated  when  they  began  development.  In  the 
case  of  Joint  Strike  Fighter,  for  example,  DOD  initially  intended  to 
purchase  2,866  aircraft  at  an  acquisition  program  unit  cost  of  about 
$66  million.  The  Navy  has  reduced  the  number  of  Joint  Strike  Fighter 
aircraft  it  plans  to  buy;  technology  and  design  problems  encountered 
during  development  have  led  to  the  significant  cost  growth.  As  a  result,  the 
acquisition  program  unit  cost  is  now  about  $84  million,  an  increase  of 
27  percent.  We  recently  reported  that  the  risk  of  even  greater  increases  is 
likely  because  flight  testing  has  not  yet  started  and  the  acquisition  strategy 
involves  substantial  overlap  of  development  and  production.  Similar 
problems  have  led  to  increases  in  the  Future  Combat  System  program.  At 
program  initiation,  the  Army  anticipated  that  each  of  15  units  would  cost 
about  $5.5  billion  to  develop  and  deliver.  Since  that  time,  instability  in  the 
program’s  technologies  and  requirements  have  led  to  significant  cost 
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increases,  leading  to  a  54  percent  increase  in  acquisition  program  unit 
cost,  now  estimated  to  be  $8.5  billion. 

Regarding  all  23  development  programs,  DOD  leaders  originally  planned  to 
invest  a  total  of  about  $83  billion  (fiscal  year  2005  dollars)  for  system 
development  and  anticipated  delivering  an  initial  operational  capability  to 
the  warfighter  in  77  months  on  average.  However,  development  costs  have 
grown  and  delivery  schedules  have  been  delayed  significantly.  DOD  now 
expects  to  invest  over  $106  billion  in  those  same  programs,  an  increase  of 
over  $23  billion  or  28  percent.  The  delivery  of  initial  capability  to  the 
warfighter  is  expected  to  take  an  average  of  88  months  or  nearly  1  year 
longer  than  originally  planned.  Figure  3  shows  changes  in  these  business 
case  elements  for  these  programs  in  the  short  time  since  their  initiation. 


Figure  3:  Cost  and  Schedule  Growth  under  DOD’s  Revised  Policy 

Cost  growth  Schedule  growth 


In  billions  of  dollars 


In  months 
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Source:  DOD  (data):  GAO  (analysis  and  presentation). 
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DOD  Is  Not 
Effectively 
Implementing  the 
Policy’s  Knowledge- 
Based,  Evolutionary 
Approach 


DOD  is  not  effectively  implementing  the  knowledge-based  process  and 
evolutionary  approach  emphasized  in  its  acquisition  policy.  While  the 
policy  outlines  a  specific  knowledge-based  process  of  concept  refinement 
and  technology  development3  to  help  ensure  a  sound  business  case  is 
developed  before  committing  to  a  new  development  program,  almost 
80  percent  of  the  programs  we  reviewed  were  permitted  to  bypass  this 
process.  Furthermore,  the  policy  emphasizes  the  need  to  mature  all  critical 
technologies  before  starting  system  development  and  to  demonstrate  that 
the  product’s  design  is  mature  before  beginning  system  demonstration. 
However,  nearly  three-fourths  of  the  programs  reported  having  immature 
critical  technologies  when  they  received  approval  to  start  development, 
and  at  least  half  of  the  programs  had  not  achieved  design  maturity  before 
holding  their  design  review  and  gaining  approval  to  enter  the  system 
demonstration  phase  of  development.  The  policy  also  emphasizes  the  use 
of  an  evolutionary  product  development  approach,  yet  program  officials 
continue  to  structure  major  acquisition  programs  to  achieve  large 
advances  in  capability  within  a  single  step  development  program.  This 
strategy  has  historically  resulted  in  poor  cost  and  schedule  outcomes. 


Knowledge-Based  Process  DOD  decision  makers  continue  to  approve  programs  for  system 

Not  Enforced  development  that  have  not  followed  key  elements  of  the  policy’s  suggested 

knowledge-based  process.  The  policy  requires  program  managers  to 
provide  senior  decision  makers  with  knowledge  about  key  aspects  of  a 
system  at  critical  investment  points  in  the  acquisition  process.  Our  prior 
reviews  have  identified  those  critical  points  as  the  start  of  system 
development  or  program  start  (referred  to  as  Milestone  B  in  the  DOD 
acquisition  policy),  design  readiness  review  separating  system  integration 
and  system  demonstration,  and  production  commitment  (Milestone  C  in 
the  DOD  acquisition  policy).  The  most  important  point  occurs  at  program 
start,  when  system  development  begins.  DOD  acquisition  guidance 
emphasizes  the  importance  of  the  acquisition  phases  preceding  program 
start,  noting  that  the  decisions  made  during  those  phases — concept 


According  to  DOD  Instruction  5000.2,  the  concept  refinement  phase  is  intended  to  refine 
the  initial  concept  and  develop  a  technology  development  strategy.  Concept  refinement 
ends  when  the  decision  authority  approves  a  preferred  solution  resulting  from  the  analysis 
of  alternatives  and  approves  the  associated  technology  development  strategy.  After 
concept  refinement,  a  project  enters  technology  development  at  Milestone  A,  when  the 
decision  maker  has  approved  the  technology  development  strategy.  The  purpose  of  this 
phase  is  to  reduce  technology  risk  and  to  determine  the  appropriate  set  of  technologies  to 
be  integrated  into  a  full  system. 
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Acquisition  Officials  Are  Not 
Effectively  Using  Early 
Processes  to  Develop 
Executable  Business  Cases 


refinement  and  technology  development — generally  define  the  nature  of 
an  entire  acquisition  program. 

Acquisition  officials  continue  to  begin  system  development  without 
following  early  processes  for  developing  executable  business  cases.  A 
business  case  should  provide  demonstrated  evidence  that  (1)  the 
warfighter’s  needs  are  real  and  necessary  and  that  they  can  best  be  met 
with  the  chosen  concept  and  (2)  the  chosen  concept  can  be  developed  and 
produced  within  existing  resources — including  technologies,  design 
knowledge,  funding,  and  the  time  to  deliver  the  product  when  it  is  needed. 
Establishing  a  business  case  calls  for  a  realistic  assessment  of  risks  and 
costs;  doing  otherwise  undermines  the  intent  of  the  business  case  and 
invites  failure.  This  process  requires  the  user  and  developer  to  negotiate 
whatever  trade-offs  are  needed  to  achieve  a  match  between  the  user’s 
requirements  and  the  developer’s  resources  before  system  development 
begins. 

The  revised  policy  and  associated  guidance  emphasize  the  importance  of 
following  a  sound  process  of  systems  engineering4  and  decision  making 
prior  to  initiating  a  system  development  program.  The  process  established 
in  the  policy  consists  of  two  phases,  concept  refinement  and  technology 
development,  and  a  major  decision  review  called  Milestone  A,  which  if 
rigorously  followed,  would  provide  acquisition  officials  with  an 
opportunity  to  assess  whether  program  officials  had  the  knowledge 
needed  to  develop  an  executable  business  case.  However,  almost 
80  percent  of  the  programs  we  reviewed  began  system  development 
without  holding  any  prior  decision  review.  Senior  officials  with  the  Office 
of  the  Secretary  of  Defense  confirmed  that  this  is  a  common  practice 
among  defense  acquisition  programs.  This  practice  eliminates  a  key 
opportunity  for  decision  makers  to  assess  early  product  knowledge 
needed  to  establish  a  business  case  that  is  based  on  realistic  cost, 
schedule,  and  performance  expectations. 

Although  program  officials  conduct  analysis  before  starting  a  development 
program,  they  do  not  consistently  follow  a  process  to  capture  the  critical 
knowledge  needed  to  produce  executable  business  cases,  as  evidenced  by 


4Systems  engineering  is  a  technical  management  tool  that  provides  the  knowledge 
necessary  to  translate  requirements  into  specific,  achievable  capabilities.  By  using  the  tools 
of  systems  engineering  during  these  early  phases  of  concept  refinement  and  technology 
development  acquisition  decision  makers  and  developers  can  work  together  to  close  gaps 
between  requirements  and  available  resources — well  before  system  development  starts. 
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Programs  Continue  to  Enter 
System  Development  with 
Immature  Technologies 


the  poor  outcomes  current  programs  are  experiencing.  Officials  with  the 
Office  of  the  Secretary  of  Defense  recognized  this  lack  of  rigor  and 
discipline  in  acquisition  process,  and  in  February  2004,  the  Under 
Secretary  of  Defense  (Acquisition,  Technology  and  Logistics)  issued  a 
department-wide  policy  memorandum  directing  acquisition  officials  to 
place  greater  emphasis  on  systems  engineering  when  planning  and 
managing  acquisition  programs.  The  policy  requires  programs  to  develop  a 
systems  engineering  plan  that  describes  the  programs’  overall  technical 
approach,  including  processes,  resources,  metrics,  and  applicable 
performance  incentives.  Although  DOD’s  systems  engineering  initiative 
has  the  potential  to  improve  program  performance,  officials  have  found 
that  the  preliminary  results  are  mixed.  Early  analysis  shows  that 
implementation  is  inconsistent  while  program  officials  learn  to  develop 
and  implement  systems-engineering  plans. 

DOD  decision  makers  continue  to  permit  programs  to  enter  system 
development  before  critical  technologies  are  mature.  Our  review  of 
technology  readiness  assessments  and  acquisition  decision  memorandums 
for  our  nine  case  study  programs  found  that  seven  of  the  nine  programs 
were  approved  to  begin  development  even  though  program  officials 
reported  levels  of  knowledge  below  the  criteria  suggested  in  the  policy 
and  associated  guidance,  specifically  in  the  area  of  technology  maturity.6 
Those  seven  programs  are  not  isolated  cases.  As  illustrated  in  Figure  4, 

13  of  the  programs  (nearly  three-fourths)  that  received  approval  to  enter 
system  development  under  the  revised  policy  did  so  with  less  than 
100  percent  of  their  critical  technologies  mature  to  the  level  specified  by 
DOD.  Only  2  of  those  programs  had  more  than  75  percent  of  their 
technologies  mature  when  they  began  (see  appendix  III  for  technology 
maturity  data  for  each  program). 


5DOD’s  revised  policy  emphasizes  the  importance  of  reducing  technology  risk  and 
demonstrating  technologies  in  a  relevant  environment  (technology  readiness  level  6)  prior 
to  program  start.  A  technology  readiness  level  of  6  means  the  technology  should  be  very 
close  to  the  planned  form,  fit,  and  function  of  its  physical  configuration  and  that  it  has  been 
tested  or  proven  to  work  in  a  relevant  environment  such  as  a  laboratory.  GAO  recommends 
a  higher  level  of  maturity  in  its  best  practice  model  based  on  best  commercial  practices. 
This  would  require  a  demonstration  of  the  technology  in  the  environment  it  is  expected  to 
be  used. 
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Figure  4:  Comparison  of  Programs  with  Mature  versus  Immature  Technologies  at 
Start  of  System  Development 


Source:  DOD  (data);  GAO  (analysis  and  presentation). 


Note:  This  figure  depicts  technology  maturity  status  for  18  of  the  23  programs  in  our  review. 
Technology  maturity  data  was  not  available  for  the  other  5  programs. 

Even  though  acquisition  policy  states  that  technologies  shall  be  mature 
before  beginning  system  development,  the  practice  of  accepting  high 
levels  of  technology  risk  at  program  start  continues  to  be  the  norm  and  not 
the  exception.  An  official  with  Office  of  the  Secretary  of  Defense 
responsible  for  reviewing  and  validating  program  assessments  of 
technology  maturity  informed  us  that  the  office  generally  views  immature 
critical  technologies  at  the  beginning  of  development  as  an  acceptable  risk 
as  long  as  program  officials  can  show  that  they  have  a  plan  to  mature  the 
technologies  by  the  time  the  program  reaches  its  design  readiness  review, 
which  requires  additional  investments  to  move  a  program  from  system 
integration  into  system  demonstration.  Therefore,  risk  management  plans 
are  consistently  viewed  as  acceptable  substitutes  for  demonstrated 
knowledge. 

Programs  Continue  Past  Design 
Reviews  before  Design 
Maturity  is  Demonstrated 


In  addition  to  emphasizing  the  importance  of  capturing  technology 
knowledge  before  starting  system  development,  DOD’s  policy  also 
highlights  the  importance  of  demonstrating  design  maturity  before  moving 
from  the  integration  phase  of  system  development  into  system 
demonstration  and  initial  manufacturing.  The  policy  establishes  a  design 
readiness  review  between  the  two  phases  to  determine  whether  a 
product’s  design  is  mature  and  stable  and  whether  the  product  is  ready  to 
move  ahead.  While  DOD’s  policy  does  not  require  programs  to 
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demonstrate  any  specific  level  of  design  maturity,  our  past  work  has  found 
that  a  key  indicator  of  design  maturity  is  the  completion  of  90  percent  of 
the  system’s  engineering  drawings.  We  found  that  defense  programs  that 
moved  forward  with  lower  levels  of  design  maturity,  as  indicated  by 
drawing  completion,  encountered  costly  design  changes  and  parts 
shortages  that,  in  turn,  caused  labor  inefficiencies,  schedule  delays,  and 
quality  problems.  Consequently,  those  programs  required  significant 
increases  in  resources — time  and  money — over  what  was  estimated  at  the 
point  each  program  entered  the  system  demonstration  phase. 

We  analyzed  engineering  drawing  completion  data  for  8  programs  initiated 
under  the  revised  policy  that  have  held  a  design  review,6  and  found  that 
more  than  half  of  those  programs  had  not  completed  90  percent  of  their 
design  drawings  before  they  received  approval  to  enter  the  system 
demonstration  phase  of  development.  We  also  analyzed  drawing-release 
data  for  three  programs  that  have  not  yet  held  their  design  review  but  have 
projected  the  number  of  drawings  officials  anticipate  will  be  completed 
when  their  reviews  are  held.  Based  on  projections  provided  by  program 
officials,  2  of  those  3  programs  are  expected  to  have  less  than  55  percent 
of  their  drawings  complete  before  they  seek  approval  to  begin  system 
demonstration  and  initial  manufacturing. 


6Some  programs  did  not  report  having  a  design  readiness  review  but  did  report  having  a 
critical  design  review.  Where  this  was  the  case,  we  assessed  those  programs’  drawing  data 
at  their  critical  design  review. 
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Table  3:  Assessment  of  Program  Design  Maturity 

Program 

Percentage  of  design  drawings 
complete  at  design  review 

Joint  Tactical  Radio  System  Cluster  1 

28% 

Global  Hawk  unmanned  aerial  vehicle 

33% 

Active  Electronically  Scanned  Array  radar 
(upgrade  for  F/A-18  E/F  fighter/attack  aircraft) 

59% 

Expeditionary  Fighting  Vehicle 

81% 

B-2  Radar  Modernization  Program 

84% 

E-2  Advanced  Hawkeye 

90% 

EA-18G 

97% 

C-5  Reliability  Enhancement  and  Re-engining 
Program 

98% 

Multi-Platform  Radar  Technology  Insertion 

Program 

100%a 

Joint  Strike  Fighter 

52  %a 

Aerial  Common  Sensor 

39%a 

Sources:  DOD  (data);  GAO  (analysis  and  presentation). 

“Program  office  projections. 


Evolutionary  Acquisition  Is  Despite  the  revised  policy’s  guidance  that  capabilities  should  be  developed 
Not  Being  Used  and  delivered  in  individually  defined  and  separately  managed  increments, 

a  majority  of  major  weapon  acquisition  programs  we  assessed  continue  to 
be  structured  to  achieve  revolutionary  increases  in  capability  within  one 
development  program.  According  to  the  policy,  the  objective  of  an 
evolutionary  approach  is  to  balance  needs  and  available  capability  with 
resources  and  put  capability  into  the  hands  of  the  user  quickly.  The  policy 
states  that  the  success  of  the  strategy  depends  on  consistent  and 
continuous  definition  of  requirements  and  the  maturation  of  technologies 
that  lead  to  disciplined  development  and  production  of  systems  that 
provide  increasing  capability.  In  this  approach,  requirements  that  cannot 
be  satisfied  within  these  limits  as  well  as  available  financial  resources 
must  wait  for  future  generations  of  the  product  and  be  managed  as 
separate  system  development  programs  with  separate  milestones,  costs, 
and  schedules.  In  our  case  studies  of  nine  acquisition  programs  initiated 
under  the  revised  policy,  we  found  only  one  program — the  Small  Diameter 
Bomb — that  satisfied  all  of  the  criteria  of  an  evolutionary  approach.  In  five 
case  studies,  we  found  that  program  officials  had  claimed  that  their 
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programs  were  evolutionary,  yet  our  evidence  shows  they  were  not 
evolutionary  in  practice;* * * * * * 7  and  in  three  cases,  program  officials  chose  not  to 
use  evolutionary  acquisition  from  the  outset.  Table  3  summarizes  our 
assessment  of  the  nine  case  studies. 


Table  4:  Assessment  of  Program  Acquisition  Strategies  for  GAO’s  Nine  Case  Studies 


Programs  in  GAO’s  case  study 

Claim  to  be  evolutionary? 

Meet  evolutionary 
criteria? 

Greater  than  30%  cost 
growth  or  more  than 

1-year  schedule  slip 

Future  Combat  System 

Yes 

No 

Yes 

Global  Hawk  (RQ-4B) 

Yes 

No 

Yes 

Joint  Strike  Fighter 

Yes 

No 

Yes 

Aerial  Common  Sensor 

Yes 

No 

Yes 

Multi-Mission  Maritime  Aircraft 

Yes 

No 

No 

Small  Diameter  Bomb 

Yes 

Yes 

No 

E-2  Advanced  Hawkeye 

No 

No 

No 

Expeditionary  Fighting  Vehicle 

No 

No 

Yes 

Multiplatform  Radar  Technology  Insertion 
Program 

No 

No 

No 

Sources:  DOD  (data);  GAO  (analysis  and  presentation). 

The  revised  acquisition  policy  does  not  contain  effective  controls  that 

require  the  demonstration  of  product  knowledge  measured  against 

specific  criteria  to  ensure  that  acquisition  officials  make  disciplined, 
transparent,  and  knowledge-based  investment  decisions.  The  lack  of 

specific  required  criteria  creates  an  environment  in  which  unknowns 

about  technology,  design,  and  manufacturing  processes  are  acceptable. 

Decision  makers  and  program  officials  are  left  with  no  objective  measures 
against  which  to  gauge  a  program’s  level  of  knowledge,  making 
accountability  difficult.  In  the  absence  of  criteria,  transparency  in 
acquisition  decisions  is  essential  to  ensuring  accountability,  but  key 
decision  documents  do  not  provide  sufficient  information  about  major 
decisions.  DOD  believes  that  acquisition  decision  memorandums,  used  to 
document  program  decisions,  provide  adequate  transparency.  However, 


Specific  Criteria  Are 
Needed  to  Ensure 
Disciplined  and 
Transparent 
Investment  Decisions 


7GAO,  TACTICAL  AIRCRAFT:  Opportunity  to  Reduce  Risks  in  the  Joint  Strike  Fighter 
Program  with  Different  Acquisition  Strategy,  GAO-05-271  (Washington,  D.C.:  March  15, 
2005)  and  GAO,  UNMANNED  AERIAL  VEHICLES:  Changes  in  Global  Hawk’s  Acquisition 
Strategy  Are  Needed  to  Reduce  Program  Risks,  GAO-05-6  (Washington,  D.C.:  November  5, 
2004). 
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the  decision  memorandums  we  reviewed  did  not  contain  an  explanation  of 
the  decision  maker’s  rationale  and  rarely  identify  remaining  risks, 
especially  as  they  relate  to  the  key  knowledge  standards  emphasized  in  the 
policy.  Further,  the  timeliness,  accessibility,  and  depth,  of  the  data 
contained  in  the  Selected  Acquisition  Reports,  DOD’s  primary  means  of 
providing  Congress  with  a  status  report  of  program  performance,  inhibits 
the  reports’  usefulness  as  a  management  and  oversight  tool. 

In  November  2003,  we  reported  that  the  revised  acquisition  policy  lacked 
many  of  the  controls  that  leading  commercial  companies  rely  on  to  attain 
an  acceptable  level  of  knowledge  before  making  additional  significant 
investments.8  Controls  are  considered  effective  if  they  are  backed  by 
specific  criteria  and  if  decision  makers  are  required  to  consider  the 
resulting  data  before  deciding  to  advance  a  program  to  the  next  level. 
Controls  used  by  leading  companies  help  decision  makers  gauge  progress 
in  meeting  cost,  schedule,  and  performance  goals  and  hold  program 
managers  accountable  for  capturing  relevant  product  knowledge  to  inform 
key  investment  decisions.  The  controls  we  have  articulated  as  best 
practices  used  by  successful  commercial  product  developers  are  listed 
below  in  table  5. 


sGAO,  DEFENSE  ACQUISITIONS:  DOD’s  Revised  Policy  Emphasizes  Best  Practices,  but 
More  Controls  Are  Needed,  GAO-04-53  (November  10,  2003). 
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Table  5:  Types  of  Controls  Considered  Best  Practices  for  Successful  Product 
Development 


Program  start  (Milestone  B):  Start  of  product  development 

Demonstrate  technologies  to  high  readiness  levels 

Ensure  that  requirements  for  the  product  are  informed  by  the  systems  engineering 
process 

Establish  cost  and  schedule  estimates  for  product  on  the  basis  of  knowledge  from 
preliminary  design  using  system  engineering  tools 

Conduct  decision  review  for  program  start 

Design  readiness  review:  Beginning  of  system  demonstration 

Complete  90  percent  of  design  drawings 

Complete  subsystem  and  system  design  reviews 

Demonstrate  with  prototype  that  design  meets  requirements 

Obtain  stakeholders’  concurrence  that  drawings  are  complete  and  producible 

Complete  the  failure  modes  and  effects  analysis 

Identify  key  system  characteristics 

Identify  critical  manufacturing  processes 

Establish  reliability  targets  and  growth  plan  on  the  basis  of  demonstrated  reliability  rates 
of  components  and  subsystems 

Conduct  decision  review  to  enter  system  demonstration 

Production  commitment  (Milestone  C):  Initiation  of  low-rate  production 

Demonstrate  manufacturing  processes 
Build  production-representative  prototypes 

Test  production-representative  prototypes  to  achieve  reliability  goal 

Test  production-representative  prototypes  to  demonstrate  product  in  operational 
environment 

Collect  statistical  process  control  data 

Demonstrate  that  critical  processes  are  capable  and  in  statistical  control 
Conduct  decision  review  to  begin  production 

Sources:  GAO  (analysis  and  presentation). 

Some  senior  officials  with  the  Office  of  the  Secretary  of  Defense  believe 
that  the  effective  use  of  controls  in  DOD’s  policy  and  the  establishment  of 
more  specific  criteria  for  decision  making  would  improve  program 
outcomes.  They  note  that  specific  criteria  need  to  be  established  and  that 
programs  need  to  be  held  accountable  to  those  criteria  before  being 
permitted  to  proceed  into  the  next  phase.  They  also  note  that  the  criteria 
for  moving  an  acquisition  effort  from  one  phase  of  the  process  to  the  next, 
primarily  documented  in  acquisition  decision  memorandums  as  exit 
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criteria,  are  not  typically  specific  and  often  do  not  relate  to  the  key 
knowledge-based  criteria  suggested  in  the  policy. 

We  found  this  to  be  true  for  our  nine  case  study  programs.  We  reviewed 
acquisition  decision  memorandums  in  our  case  studies  and  determined 
that  they  were  not  useful  in  explaining  the  decision  maker’s  rationale  and 
in  almost  all  of  the  cases  they  did  not  address  the  key  knowledge  criteria 
suggested  in  the  acquisition  policy.  In  most  instances,  the  decision  maker 
simply  noted  that  the  program  being  assessed  was  ready  to  proceed  into 
system  development,  but  did  not  provide  an  explanation  of  the  rationale 
for  the  decision.  Senior  officials  with  the  Office  of  the  Secretary  of 
Defense  told  us  that  they  agree  that  a  better  explanation  of  the  decision 
maker’s  rationale,  specifically  in  instances  where  the  knowledge  criteria 
are  not  fully  met,  would  provide  transparency  and  ultimately  allow  for  a 
more  accountable  decision-making  process.  The  following  two  examples 
illustrate  how  decision  documentation  is  lacking: 

•  The  Future  Combat  System  program  received  approval  to  enter  system 
development  and  demonstration  in  2003,  with  19  percent  of  its  critical 
technologies  mature,  well  below  the  policy’s  standard.  The  acquisition 
decision  memorandum  supporting  this  decision  did  not  provide  the 
rationale  for  approving  the  system  with  such  a  large  number  of  immature 
critical  technologies.  The  memo  did  direct  an  updated  review  of  the 
decision  18  months  later  and  that  the  program  “remain  flexible  and  open 
to  accommodate  trades  in  the  system  architecture  and  in  the  individual 
systems’  designs.” 

•  The  Joint  Strike  Fighter  program  was  approved  to  enter  system 
development  in  2001.  The  acquisition  decision  memorandum  did  not 
address  the  fact  that  75  percent  of  the  program’s  critical  technologies  were 
not  mature  to  the  policy’s  standard.  The  memorandum  did  acknowledge 
that  the  program’s  requirements  could  be  changed  or  modified,  noting  that 
further  refinements  in  the  requirements  should  be  explored  as  a  potential 
way  to  reduce  program  costs.  However,  the  memorandum  did  not  explain 
why  the  decision  maker  determined  that  the  program  should  enter 
development  without  achieving  the  technology  and  requirements 
knowledge  emphasized  in  the  policy. 

The  acquisition  decision  memorandums  for  most  of  the  other  programs  we 
reviewed  did  not  specifically  address  critical  gaps  in  knowledge,  nor  did 
they  effectively  explain  the  decision  makers’  rationale  for  deeming  those 
programs  ready  to  begin  system  development.  In  memos  where  we  found  a 
reference  to  key  knowledge  principles,  such  as  technology  maturity,  the 
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decision  makers  acknowledged  that  more  effort  was  needed  to  meet  the 
policy’s  suggested  criteria  but  considered  the  risk  acceptable  to  begin 
development.  These  memos  did  not  explain  why  risks  were  considered 
acceptable.  For  example,  the  Navy’s  Multi-Mission  Maritime  Aircraft 
program  had  none  of  its  critical  technologies  mature  at  program  initiation. 
The  decision  maker  acknowledged  the  need  to  further  mature  the  critical 
technologies  but  approved  the  program  to  enter  development.  Instead  of 
holding  the  program  to  the  policy’s  criteria  for  entering  development,  the 
decision  maker  simply  directed  the  Navy  to  work  with  the  Office  of  the 
Secretary  of  Defense  to  implement  risk  mitigation  and  technology 
maturation  plans  during  the  integration  phase  of  system  development. 

In  addition  to  the  lack  of  transparency  provided  through  acquisition 
decision  memoranda,  we  also  found  that  the  data  presented  to  Congress  in 
DOD’s  Selected  Acquisition  Reports  ( SARs )  provided  only  limited 
usefulness  as  an  oversight  tool.  Since  1969,  SARs  have  been  the  primary 
means  by  which  DOD  reports  the  status  of  major  weapon  system 
acquisitions  to  Congress.  SARs  are  reports  that  are  expected  to  contain 
information  on  the  cost,  schedule,  and  performance  of  major  weapon 
systems  in  comparison  with  baseline  values  established  at  program  start, 
full-scale  development,  and  production  decision  points.  Our  analysis,  as 
well  as  a  previous  GAO  review,9  of  current  and  historical  SAR  data  found 
that  the  timeliness,  accessibility,  and  depth  of  the  data  contained  in  the 
reports  limits  their  usefulness  as  an  oversight  tool.  Our  prior  review  noted 
that  a  number  of  opportunities  exist  for  DOD  to  give  Congress  more 
complete  information  on  the  performance  of  major  defense  acquisition 
programs.  DOD  agreed  that  SAR  data  could  be  improved  to  make  it  more 
useful  to  Congress. 


Conclusions 


Failing  to  consistently  implement  the  knowledge-based  process  and 
evolutionary  principles  emphasized  in  the  revised  acquisition  policy — 
coupled  with  a  lack  of  specific  criteria  for  making  key  investment 
decisions — are  keeping  DOD  on  its  historical  path  of  poor  cost  and 
schedule  outcomes.  Most  programs  are  incurring  the  same  scope  of  cost 
overruns  and  schedule  delays  as  programs  managed  under  prior  DOD 
policies.  More  consistent  use  of  the  early  acquisition  processes  would 
improve  the  quality  and  viability  of  program  business  cases  by  ensuring 


9GAO,  DEFENSE  ACQUISITIONS:  Inform  ation  for-  Congress  on  Performance  of  Major 
Programs  Can  Be  More  Complete,  Timely,  and  Accessible,  GAO-05-182  (March  28,  2005). 
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they  are  founded  on  knowledge  obtained  from  rigorous  and  disciplined 
analysis.  The  initiative  by  Office  of  the  Secretary  of  Defense  to  reinstitute 
the  use  of  systems  engineering  is  a  step  in  the  right  direction.  However,  in 
order  for  this  initiative  to  be  effective  DOD  must  establish  and  enforce 
specific  criteria  at  key  decision  points.  Our  past  work  has  identified  and 
recommended  criteria  and  controls  that  should  be  consistently  applied  at 
major  decision  points.  The  enforcement  of  these  criteria  is  critical  to 
ensuring  that  programs  have  the  knowledge  necessary  to  successfully 
move  forward  through  the  acquisition  process.  DOD  officials  have 
acknowledged  the  advantages  of  using  knowledge-based  criteria  and 
controls,  but  believe  the  policy  already  includes  enough  controls  to 
achieve  effective  program  results.  However,  without  enforceable  criteria, 
defense  officials  are  challenged  to  determine  whether  adequate  knowledge 
has  been  obtained  for  investing  taxpayer  dollars.  The  lack  of  enforceable 
criteria  also  makes  it  difficult  to  hold  defense  officials  accountable  for 
their  decisions. 


Recommendations 
for  Executive  Action 


DOD  must  ensure  that  appropriate  knowledge  is  captured  and  used  at 
critical  junctures  to  make  decisions  about  moving  a  program  forward  and 
investing  more  money.  We  recommend  that  the  Secretary  of  Defense 
require  program  officials  to  demonstrate  that  they  have  captured 
appropriate  knowledge  at  three  key  points — program  start,  design  review 
for  transitioning  from  system  integration  to  system  demonstration,  and 
production  commitment — as  a  condition  for  investing  resources.  At  a 
minimum  those  controls  should  require  program  officials  to  demonstrate 
that  they  have  achieved  a  level  of  knowledge  that  meets  or  exceeds  the 
following  criteria  at  each  respective  decision  point: 


•  Program  start  (Milestone  B):  Start  of  product  development 

•  Demonstrate  technologies  to  high  readiness  levels 

•  Ensure  that  requirements  for  the  product  are  informed  by  the  systems- 
engineering  process 

•  Establish  cost  and  schedule  estimates  for  product  on  the  basis  of 
knowledge  from  preliminary  design  using  system  engineering  tools 

•  Conduct  decision  review  for  program  start 
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•  Design  readiness  review:  Beginning  of  system  demonstration 

•  Complete  90  percent  of  design  drawings 

•  Complete  subsystem  and  system  design  reviews 

•  Demonstrate  with  prototype  that  design  meets  requirements 

•  Obtain  stakeholders’  concurrence  that  drawings  are  complete  and 
producible 

•  Complete  the  failure  modes  and  effects  analysis 

•  Identify  key  system  characteristics 

•  Identify  critical  manufacturing  processes 

•  Establish  reliability  targets  and  growth  plan  on  the  basis  of 
demonstrated  reliability  rates  of  components  and  subsystems 

•  Conduct  decision  review  to  enter  system  demonstration 

•  Production  commitment  (Milestone  C):  Initiation  of  low-rate 
production 

•  Demonstrate  manufacturing  processes 

•  Build  production-representative  prototypes 

•  Test  production-representative  prototypes  to  achieve  reliability  goal 

•  Test  production-representative  prototypes  to  demonstrate  product  in 
operational  environment 

•  Collect  statistical  process  control  data 

•  Demonstrate  that  critical  processes  are  capable  and  in  statistical 
control 

•  Conduct  decision  review  to  begin  production 

Furthermore,  to  ensure  that  major  decisions  are  transparent  and  that  program 
officials  and  decision  makers  are  held  accountable,  we  recommend  that  the 
Secretary  of  Defense  require  decision  makers  to  include  written  rationale  for 
each  major  decision  in  acquisition  decision  documentation.  The  rationale 
should  address  the  key  knowledge-based  criteria  appropriate  for  milestone 
decisions,  explain  why  a  program’s  level  of  knowledge  in  each  area  was 
deemed  acceptable  if  criteria  have  not  been  met  and  provide  a  plan  for 
achieving  the  knowledge  necessary  to  meet  criteria  within  a  given  time  frame. 


DOD  provided  written  comments  on  a  draft  of  this  report.  The  comments 
appear  in  appendix  II. 

DOD  partially  concurred  with  our  recommendation  that  the  Secretary  of 
Defense  should  establish  specific  controls  to  insure  that  program  officials 
demonstrate  that  they  have  captured  a  level  of  knowledge  that  meets  or 
exceeds  specific  criteria  at  three  key  points  in  the  acquisition  process: 
program  start,  design  readiness  review,  and  production  commitment.  DOD 
agreed  that  knowledge-based  decision  making  is  consistent  with  sound 
business  practice  and  stated  that  it  would  continue  to  develop  policy  that 


Agency  Comments 
and  Our  Evaluation 
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reflects  a  knowledge-based  approach  and  improves  acquisition  outcomes. 
DOD  noted  that  it  would  consider  our  recommendations  as  it  reassesses 
the  DOD  acquisition  business  model  and  the  knowledge  required  at  each 
decision  point.  We  believe  that  DOD’s  plan  to  reassess  its  business  model 
provides  a  good  opportunity  to  establish  the  controls  and  specific  criteria 
recommended  in  this  report.  Therefore,  we  are  retaining  our 
recommendation  that  the  Secretary  of  Defense  should  establish  controls 
to  insure  that  program  officials  demonstrate  that  they  have  captured  a 
level  of  knowledge  that  meets  or  exceeds  specific  criteria  at  three  key 
points  in  the  acquisition  process. 

DOD  also  partially  concurred  with  our  recommendation  that  the  Secretary 
of  Defense  require  decision  makers  to  provide  written  rationale  in 
acquisition  decision  documentation  for  each  major  decision.  DOD  agreed 
that  acquisition  decisions  should  be  documented,  decision  makers  should 
be  held  accountable,  and  that  they  should  provide  the  rationale  for  their 
decisions.  DOD  believes  that  the  implementation  of  Section  801  of  the 
National  Defense  Authorization  Act  for  FY  2006  reinforces  these 
processes.  The  act  calls  for  the  decision  maker  to  certify  that  the  program 
meets  certain  requirements,  such  as  technology  maturity,  prior  to  starting 
a  new  development  program  at  Milestone  B.  However,  the  act  is  focused 
on  the  decision  to  start  a  development  program  and  does  not  identify 
specific  criteria  for  programs  to  be  measured  against  at  design  readiness 
review  or  production  commitment.  We  believe  our  recommendation  adds 
transparency  and  accountability  to  the  process  because  it  requires  the 
decision  maker  to  provide  the  rationale  for  a  decision  to  allow  a  program 
to  move  forward,  not  only  at  Milestone  B  but  at  other  key  decision  points 
as  well.  Therefore,  we  are  retaining  our  recommendation  that  the 
Secretary  of  Defense  require  decision  makers  to  provide  written  rationale 
for  each  major  decision  in  acquisition  decision  documentation. 


We  are  sending  copies  of  this  report  to  the  Secretary  of  Defense;  the 
Secretaries  of  the  Air  Force,  Army,  and  Navy;  and  the  Director  of  the 
Office  of  Management  and  Budget.  We  will  provide  copies  to  others  on 
request.  This  report  will  also  be  available  at  no  charge  on  GAO’s  Web  site 
at  http://www.gao.gov. 
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If  you  have  any  questions  about  this  report  or  need  additional  information, 
please  call  me  at  (202)  512-4841  (sullivanm@gao.gov).  Contact  points  for 
the  offices  of  Congressional  Relations  and  Public  Affairs  are  located  on 
the  last  page  of  this  report.  Key  contributors  to  this  report  were  Michael 
Hazard,  Assistant  Director;  Lily  Chin;  Ryan  Consaul;  Christopher  DePerro; 
Travis  Masters;  and  Adam  Vodraska. 


Michael  J.  Sullivan 
Director,  Acquisition 
and  Sourcing  Management 
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Appendix  I:  Scope  and  Methodology 


To  assess  the  impact  of  DOD’s  revised  acquisition  policy,  we  analyzed  cost 
and  schedule  data  for  23  major  defense  acquisition  programs  that  were 
approved  to  begin  system  development  under  the  revised  policy.  We  did 
not  assess  space,  missile  defense,  or  ship  programs.  We  collected  our  data 
from  Selected  Acquisition  Reports,  presidential  budget  documents, 
ongoing  GAO  work,  and  pertinent  program  officials.  We  utilized  previous 
GAO  reports  related  to  defense  acquisition  policies  and  worked  with 
knowledgeable  GAO  staff  to  ensure  the  use  of  current,  accurate  data.  We 
also  analyzed  more  than  150  annual  Selected  Acquisition  Reports  covering 
a  36-year  period  from  1969  to  2005,  to  determine  historical  trends  related 
to  outcomes  of  acquisition  policy  implementation. 

We  assessed  whether  the  revised  policy’s  knowledge-based,  evolutionary 
acquisition  principles  were  being  effectively  implemented  by  conducting 
9  case  study  reviews  and  analyzing  design  maturity  data  for  11  programs 
that  have  made  engineering-drawing  data  available  to  GAO.  Our  case  study 
programs  were  the  Aerial  Common  Sensor,  Multi-Platform  Radar 
Technology  Insertion  Program,  Global  Hawk  Unmanned  Aerial  Vehicle, 
Small  Diameter  Bomb,  Future  Combat  System,  Joint  Strike  Fighter, 
Expeditionary  Fighting  Vehicle,  Multi-Mission  Maritime  Aircraft,  and  the 
E-2  Advanced  Hawkeye.  We  interacted  directly  with  numerous  program 
officials  to  seek  input  on  current  developments  with  their  programs.  We 
studied  program  documents  to  assess  how  well  programs  understand  and 
are  implementing  the  revised  acquisition  policy.  We  also  analyzed  drawing 
release  data  for  those  programs  that  have  either  passed  their  design 
review  or  have  provided  GAO  with  estimated  drawing  release  data  for  a 
future  design  review  to  assess  design  maturity.  In  several  cases,  we  asked 
that  program  offices  verify  information  in  these  various  documents. 

We  also  reviewed  Department  of  Defense  (DOD)  Directive  5000.1,  DOD 
Instruction  5000.2,  and  the  Defense  Acquisition  Guidebook.  In  addition  we 
examined  each  of  the  military  services’  policy  directives  and  guidance, 
DOD  memorandums  to  include  policy  intent  and  DOD  expectations 
regarding  policy  implementation  as  well  as  Joint  Capabilities  Integration 
and  Development  System  documents.  We  interviewed  relevant  officials  in 
Washington,  D.C.,  from  the  Office  of  the  Director,  Defense  Research  and 
Engineering,  the  Joint  Staff,  the  Office  of  the  Secretary  of  Defense,  and 
Army,  Navy,  and  Air  Force  acquisition  policy  staff  in  order  to  better 
understand  the  content  of  these  documents  and  the  intent  of  DOD’s  policy. 

We  conducted  our  review  from  May  2005  to  February  2006  in  accordance 
with  generally  accepted  government  auditing  standards. 
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Appendix  II:  Comments  from  the  Department 
of  Defense 


OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 
3000  DEFENSE  PENTAGON 
WASHINGTON.  DC  20301-3000 


APR  1  3  2006 


DPAP/PAIC 

Mr.  Michael  Sullivan 

Director,  Acquisition  and  Sourcing  Management 
U.S.  Government  Accountability  Office 
441  G  Street,  N.W. 

Washington,  D.C.  20548 

Dear  Mr.  Sullivan: 

This  is  the  Department  of  Defense  response  to  the  GAO  draft  report,  "DEFENSE 
ACQUISITIONS:  Major  Weapon  Systems  Continue  to  Experience  Cost  and  Schedule 
Problems  under  DOD's  Revised  Policy"  (GAO-06-368/GAO  Code  120448)  DOD 
Acquisition  Policy  Implementation.  I  concur  with  the  recommendation  in  the  report. 
Additional  comment  is  attached. 


ACQUISITION. 
TECHNOLOGY 
AND  LOGISTICS 


Sincerely, 


ShawA&ad 

Director:  Defense  Procurement 
and  Acquisition  Policy 


Attachment: 
As  stated 
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Appendix  II:  Comments  from  the  Department 
of  Defense 


GAO  DRAFT  REPORT  -  DATED  MARCH  7,  2006 
GAO  CODE  120448  /GAO-06-368 

"DEFENSE  ACQUISITIONS:  Major  Weapon  Systems  Continue  to  Experience 
Cost  and  Schedule  Problems  under  DOD's  Revised  Policy" 

DEPARTMENT  OF  DEFENSE  COMMENT 
ON  THE  RECOMMENDATION 

GAO  RECOMMENDATION: 

DOD  must  ensure  that  appropriate  knowledge  is  captured  and  used  at  critical  junctures  to 
make  decisions  about  moving  a  program  forward  and  investing  more  money.  We 
recommend  that  the  Secretary  of  Defense  require  program  officials  to  demonstrate  that 
they  have  captured  appropriate  knowledge  at  three  key  points  —  program  start,  design 
review  for  transitioning  from  system  integration  to  system  demonstration,  and  production 
commitment  —  as  a  condition  for  investing  resources.  At  a  minimum  those  controls 
should  require  program  officials  to  demonstrate  that  they  have  achieved  a  level  of 
knowledge  that  meets  or  exceeds  the  following  criteria  at  each  respective  decision  point: 

Program  start  (Milestone  B):  Start  of  product  development 

•  Demonstrate  technologies  to  high  readiness  levels 

•  Ensure  that  requirements  for  the  product  are  informed  by  the  systems  engineering 
process 

•  Establish  cost  and  schedule  estimates  for  product  on  the  basis  of  knowledge  from 
preliminary  design  using  system  engineering  tools 

•  Conduct  decision  review  for  program  start 

Design  readiness  review:  Beginning  of  system  demonstration 

•  Complete  90  percent  of  design  drawings 

•  Complete  subsystem  and  system  design  reviews 

•  Demonstrate  with  prototype  that  design  meets  requirements 

•  Obtain  stakeholders’  concurrence  that  drawings  are  complete  and  producible 

•  Complete  the  failure  modes  and  effects  analysis 

•  Identify  key  system  characteristics 

•  Identify  critical  manufacturing  processes 

•  Establish  reliability  targets  and  growth  plan  on  the  basis  of  demonstrated  reliability 
rates  of  components  and  subsystems 

•  Conduct  decision  review  to  enter  system  demonstration 
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Appendix  II:  Comments  from  the  Department 
of  Defense 


Production  commitment  (Milestone  C):  Initiation  of  low-rate  production 

•  Demonstrate  manufacturing  processes 

•  Build  production-representative  prototypes 

•  Test  production-representative  prototypes  to  achieve  reliability  goal 

•  Test  production-representative  prototypes  to  demonstrate  product  in  operational 
environment 

•  Collect  statistical  process  control  data 

•  Demonstrate  that  critical  processes  are  capable  and  in  statistical  control 

•  Conduct  decision  review  to  begin  production 

Furthermore,  to  ensure  that  major  decisions  are  transparent  and  that  program  officials  and 
decision  makers  are  held  accountable,  we  recommend  that  the  Secretary  of  Defense 
require  decision  makers  to  include  written  rationale  for  each  major  decision  in  acquisition 
decision  documentation.  The  rationale  should  address  the  key  knowledge-based  criteria 
appropriate  for  milestone  decisions,  explain  why  a  program’s  level  of  knowledge  in  each 
area  was  deemed  acceptable  if  criteria  have  not  been  met,  and  provide  a  plan  for 
achieving  the  knowledge  necessary  to  meet  criteria  within  a  given  time  frame. 

DOD  RESPONSE:  Partial  Concurrence 

Partially  Concur.  The  Department  of  Defense  agrees  that  knowledge-based  decision 
making  is  consistent  with  sound  business  practice.  Consequently,  we  will  continue  to 
develop  policy  that  reflects  that  approach  and  improves  acquisition  outcomes.  In  keeping 
with  that  objective  and  in  association  with  the  results  of  the  recently  completed 
Quadrennial  Defense  Review,  we  will  soon  be  reassessing  the  DoD  acquisition  business 
model  and  the  knowledge  required  at  each  decision  point.  In  that  vein,  we  will  consider 
the  GAO  proposals  during  that  process.  We  also  agree  that  acquisition  decisions  should 
be  documented,  that  decision  makers  should  be  held  accountable,  and  that  they  should 
provide  the  rationale  for  their  decisions — a  process  reinforced  through  implementation  of 
Section  801  of  the  National  Defense  Authorization  Act  for  FY  2006. 
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Appendix  III:  Program  Data  for  23  Programs 
Initiated  under  DOD’s  Revised  Acquisition 
Policy  (as  of  December  2005) 


Formal 

Percent 

Percent 

Percent 

Percent 

Milestone  T  or 

technology 

design 

growth  in 

growth  in 

Milestone  A 

mature  (TRL 

drawings 

estimated 

estimated 

Program 

decision 

6)  at  program 

complete  at 

development 

development 

Program 

start 

review? 

start 

design  review 

cost0 

schedule 

Expeditionary  Fighting  Vehicle 

12/2000 

Yes 

80% 

81% 

61% 

70% 

Active  Electronically  Scanned 
Array  radar  (upgrade  for  F/A-18 
E/F  fighter/attack  aircraft) 

12/2000 

No 

0% 

59% 

14% 

1% 

Global  Hawk  unmanned  aerial 
vehicle 

2/2001 

No 

0% 

33% 

166% 

Undetermined 

UH-60M  helicopter  upgrade 

4/2001 

No 

Not  available 

Not  available 

151% 

25% 

C-130  Avionics  Modernization 
Program 

8/2001 

No 

100% 

Not  available 

122% 

Undetermined 

Joint  Strike  Fighter 

10/2001 

Yes 

25% 

52%b 

30% 

23% 

C-5  Reliability  Enhancement  and 
Re-engining  Program 

11/2001 

Yes 

100% 

98% 

0% 

25% 

Joint  Tactical  Radio  System 
Cluster  1 

6/2002 

No 

0% 

28% 

31% 

44% 

Joint  Tactical  Radio  System 
Waveform 

6/2002 

No 

Not  available 

Not  available 

44% 

Undetermined 

Advanced  Anti-radiation  Guided 
Missile 

4/2003 

No 

Not  available 

Not  available 

7% 

0% 

Multi-Platform  Radar  Technology 
Insertion  Program 

4/2003 

No 

100% 

100%  b 

0% 

Undetermined 

Future  Combat  System 

5/2003 

No 

19% 

Not  available 

48% 

53% 

E-2  Advanced  Hawkeye 

6/2003 

No 

50% 

90% 

5% 

0% 

Warfighter  Information  Network- 
Tactical 

7/2003 

No 

25% 

Not  available 

0% 

0% 

Small  Diameter  Bomb 

10/2003 

Yes 

100% 

Not  available 

0% 

0% 

EA-18G 

11/2003 

No 

60% 

97% 

7% 

0% 

Joint  Tactical  Radio  System 
Cluster  5 

4/2004 

No 

50% 

Not  available 

0% 

2% 

Multi-Mission  Maritime  Aircraft 

5/2004 

No 

0% 

Not  available 

0% 

0% 

Standard  Missile-6  Extended 
Range  Active  Missile  Block  1 

6/2004 

No 

Not  available 

Not  available 

0% 

0% 

Aerial  Common  Sensor 

7/2004 

Yes 

50% 

39%  b 

45% 

36% 

B-2  Radar  Modernization 

Program 

7/2004 

No 

100% 

84% 

0% 

0% 

Patriot/Medium  Extended  Air 
Defense  System  Combined 
Aggregate  Program  (fire  unit) 

8/2004 

No 

83% 

Not  available 

0% 

0% 

Mission  Planning  System 

12/2004 

No 

Not  available 

Not  available 

0% 

0% 
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Appendix  III:  Program  Data  for  23  Programs 
Initiated  under  DOD’s  Revised  Acquisition 
Policy  (as  of  December  2005) 


Sources:  DOD  (data};  GAO  {analysis  and  presentation). 

Note:  In  this  table  the  term  “not  available”  means  that  GAO  had  not  received  sufficient  data  to  make 
an  assessment  of  the  given  program's  design  and/or  technology  maturity. 

“Milestone  I  was  a  forerunner  to  Milestone  A,  the  decision  review  that  currently  precedes  the  start  of 
technology  development. 

bProgram  office  projections. 

“Cost  growth  is  expressed  as  the  percent  change  in  program  development  cost  estimates  in  fiscal 
year  2005  dollars. 
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Related  GAO  Products 


DOD  Acquisition  Outcomes:  A  Case  for  Change.  GAO-06-257T. 
Washington,  D.C.:  November  15,  2005. 

Defense  Acquisitions:  Stronger  Management  Practices  Are  Needed  to 
Improve  DOD’s  Software-Intensive  Weapon  Acquisitions.  GAO-04-393. 
Washington,  D.C.:  March  1,  2004. 

Best  Practices:  Setting  Requirements  Differently  Could  Reduce  Weapon 
Systems’  Total  Ownership  Costs.  GAO-03-57.  Washington,  D.C.:  February 
11,  2003 

Best  Practices:  Capturing  Design  and  Manufacturing  Knowledge  Early 
Improves  Acquisition  Outcomes.  GAO-02-701.  Washington,  D.C.:  July  15, 
2002. 

Defense  Acquisitions:  DOD  Faces  Challenges  in  Implementing  Best 
Practices.  GAO-02-469T.  Washington,  D.C.:  February  27,  2002. 

Best  Practices:  Better  Matching  of  Needs  and  Resources  Will  Lead  to 
Better  Weapon  System  Outcomes.  GAO-Ol-288.  Washington,  D.C.:  March  8, 
2001. 

Best  Practices:  A  More  Constructive  Test  Approach  Is  Key  to  Better 
Weapon  System  Outcomes.  GAO/NSIAD-OO-199.  Washington,  D.C.:  July  31, 
2000. 

Defense  Acquisition:  Employing  Best  Practices  Can  Shape  Better 
Weapon  System  Decisions.  GAO/T-NSIAD-OO-137.  Washington,  D.C.:  April 
26,2000.  ~ 

Best  Practices:  DOD  Training  Can  Do  More  to  Help  Weapon  System 
Programs  Implement  Best  Practices.  GAO/NSIAD-99-206.  Washington, 
D.C.:  Augustl6,  1999. 

Best  Practices:  Better  Management  of  Technology  Development  Can 
Improve  Weapon  System  Outcomes.  GAO/NSIAD-99-162.  Washington, 

D.C.:  July  30,  1999. 

Defense  Acquisitions:  Best  Commercial  Practices  Can  Improve  Program 
Outcomes.  GAO/T-NSIAD-99-116.  Washington,  D.C.:  March  17,  1999. 

Defense  Acquisition:  Improved  Program  Outcomes  Are  Possible.  GAO/T- 
NSLAD-98-123.  Washington,  D.C.:  March  17,  1998. 
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Related  GAO  Products 


Best  Practices:  DOD  Can  Help  Suppliers  Contribute  More  to  Weapon 
System  Programs.  GAO/NSIAD-98-87.  Washington,  D.C.:  March  17,  1998. 

Best  Practices:  Successful  Application  to  Weapon  Acquisition  Requires 
Changes  in  DOD’s  Environment.  GAO/NSIAD-98-56.  Washington,  D.C.: 
February  24,  1998. 

Best  Practices:  Commercial  Quality  Assurance  Practices  Offer 
Improvements  for  DOD .  GAO/NSIAD-96-162.  Washington,  D.C.:  August  26, 
1996. 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and  policies; 
and  provides  analyses,  recommendations,  and  other  assistance  to  help 
Congress  make  informed  oversight,  policy,  and  funding  decisions.  GAO’s 
commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no  cost 
is  through  GAO’s  Web  site  (www.gao.gov).  Each  weekday,  GAO  posts 
newly  released  reports,  testimony,  and  correspondence  on  its  Web  site.  To 
have  GAO  e-mail  you  a  list  of  newly  posted  products  every  afternoon,  go 
to  www.gao.gov  and  select  “Subscribe  to  Updates.” 

Order  by  Mail  or  Phone 

The  first  copy  of  each  printed  report  is  free.  Additional  copies  are  $2  each. 

A  check  or  money  order  should  be  made  out  to  the  Superintendent  of 
Documents.  GAO  also  accepts  VISA  and  Mastercard.  Orders  for  100  or 
more  copies  mailed  to  a  single  address  are  discounted  25  percent.  Orders 
should  be  sent  to: 

U.S.  Government  Accountability  Office 

441  G  Street  NW,  Room  LM 

Washington,  D.C.  20548 

To  order  by  Phone:  Voice:  (202)  512-6000 

TDD:  (202)  512-2537 

Fax:  (202)  512-6061 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Web  site:  www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Gloria  Jarmon,  Managing  Director,  JarmonG@gao.gov  (202)  512-4400 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7125 
Washington,  D.C.  20548 

Public  Affairs 


Paul  Anderson,  Managing  Director,  AndersonPl@gao.gov  (202)  512-4800 
U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 
Washington,  D.C.  20548 
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